Communication-Efficient Distributed SGD with Error-Feedback, Revisited

We show that the convergence proof of a recent algorithm called dist-EF-SGD for distributed stochastic gradient descent with communication efficiency using error-feedback Zheng et al. (NeurIPS 2019) is problematic mathematically. Concretely, original error bound arbitrary sequences learning rate unfortunately incorrect, leading to an invalidated upper in theorem algorithm. As evidences, we explicitly provide several counter-examples, both convex and non-convex cases, incorrectness bound. fix issue by providing new its corresponding proof, algorithm, therefore recovering mathematical analysis.